Reproducing apparatus, reproducing method and reproducing computer program product

ABSTRACT

An apparatus, computer program product and method provide a mechanism for independently allow a user to pause the display of a video stream from a given source, while also enabling the user to operate a menu function from a data stream from the same source.

TECHNICAL FIELD

This invention is related to a reproducing apparatus, a reproducingmethod and a reproducing computer program product to improve anefficiency of a user interactive operation against to high capacityrecording medium, such as BLU-RAY DISC™.

BACKGROUND ART

Recently, the BLU-RAY disc format has been developed as an attachabledisc recordable medium format. The BLU-RAY disc format uses a discrecording medium whose diameter is 12 cm, cover layer of 0.1 mm, andblue-violet laser whose wave length is 405 nm. Its capacity is about 25GB. By using this format, it is possible to record a high definitiontelevision signal for more than 2 hours without deteriorating the image.

It is assumed that the source of AV (Audio/Video) signal recorded onthis recordable medium can be an analog video signal and a digitaltelevision signal. The BLU-RAY format to record these AV signals hasbeen already developed.

On the other hand, a read only medium format, which is a format todefine a previously recorded movie or music on a media, is still underdevelopment. Although DVD (digital versatile disc) is widely known andpopular as a disc to record movies and music, the BLU-RAY disc issuperior to DVD in that it is possible to record high definitiontelevision signals of more than two hours by its high capacity and hightransfer rate.

A “menu” is often recorded with content to control execution of variousprograms relating to content when selling package medium recordedcontents, such as a movie. For example, the menu is displayed bypressing a remote control commander button, and a user selects the menuitem using an arrow key. Then play-back device then executes apredetermined process (or operation) corresponding to the menu itemselected on the remote commander.

A BLU-RAY read only disc player will display a menu when the user inputsa user operation while playing contents. That is to say, while playingcontents, the menu is displayed on top of the playing contents (e.g.,movie, video, or image) by the user pressing a predetermined button onthe remote commander or play-back unit. A technique to keep playingcontents while displaying a menu is disclosed in patent documentsJP2005-222683.

As recognized by the present inventors, this reproducing apparatus (orplay-back unit) controls the display of the menu and the contents usinga same time axis. Therefore, for example, when the user makes the movietemporally stop (i.e., pause), the menu that is displayed on top of thecontent is also paused and so user operations (menu selections) areneither accepted nor executed.

By way of example, the present inventors recognized the situation wherea user can change a spoken language (eg. Japanese and English) and asubtitle (eg. Japanese and English) by using the displayed menu whileplaying the contents. For example, when a user try to change a spokenlanguage and a subtitle while the content is being played, the user isforced to stop watching the content in order to change the language andsubtitle.

On the other hand, after displaying the menu, the user cannot change thespoken language and subtitle because the displayed menu also stops asdescribed above. Therefore after the user changes the spoken languageand subtitle while the content is being played, the user needs toreverse the video so as to watch the portion of the content that he waspreviously watching prior to changing the language and subtitlesettings.

DISCLOSURE OF THE INVENTION

One aspect of the invention is to provide a reproducing apparatus,reproducing method and reproducing computer program product that makesit possible for a user to operate a displayed menu regardless if themain content is being actively played or paused.

The present invention is a reproducing apparatus comprising a referencesignal unit configured to output a reference signal; an acquiring unitconfigured to acquire a first stream of data that represents a mainimage to be reproduced on a display under control of said referencesignal, a second stream of data that represents a menu to be displayedwith said main image, said second stream being asynchronous with saidfirst stream, and said menu including an item that is user-selectable;and a reproducing control information; an operation receiving unit toreceive an operation request when the item in said menu is selected; anda reproducing control unit configured to control reproduction of saidmain image from said first stream, and menu from said second streambased on said reproducing control information, wherein said reproducingcontrol unit is configured to control the operation receiving unit toexecute a computer-implemented operation when the operation request inresponse to the item in said menu being selected even while reproductionof the main image is paused.

Consequently, a user is able to select displayed menu items using basedon a second stream of data while the reproduction of content from afirst stream of data is paused.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a descriptive diagram of an abstract of a BD-ROM data model.

FIG. 2 is a diagram of an abstract of an index table.

FIG. 3 is a logic diagram showing a relationship between clipinformation, clip, PlayItem, and PlayList.

FIG. 4 is a flow diagram showing how one clip relates to a plurality ofPlayLists.

FIG. 5 is a diagram of a Subpath.

FIG. 6 shows a file management structure recorded on a medium.

FIG. 7 is a syntax figure showing syntax of MoviePlayList file (PlayListfile) “xxxxx.mpls”.

FIG. 8 shows an example syntax of PlayList( ) block.

FIGS. 9A and 9B show an example syntax of PlayItem( ) block.

FIGS. 10A, 10B and 10C show an example syntax of a first part of aSTN_table( ) block.

FIGS. 11A and 11B show another portion of the example syntax of theSTN_table( ) block.

FIG. 12 shows an example of syntax of Stream_entry( ) block.

FIG. 13 shows an example of syntax of SubPath( ) block.

FIGS. 14A and 14B show an example of syntax of SubPlayItem( ) block.

FIGS. 15A and 15B are table of subpath_types defined in SubPath( )block.

FIGS. 16A-16D show an example recording format of a button image.

FIG. 17 shows an example syntax of an example of ICS.

FIG. 18 shows an example syntax ofinteractive_composition_data_fragment( ) block.

FIGS. 19A and 19B are flow charts to indicate BD virtual playeroperation.

FIG. 20 shows a flow chart to indicate BD virtual player operation.

FIG. 21 shows an example of plane structure used as display system ofimage.

FIG. 22 is a block diagram indicating how to mix moving picture plane,subtitle plane and graphics plane.

FIG. 23 is an exemplary palette table.

FIG. 24 shows user interface illustrating how a displaying menu based onIG stream and video stream based on PlayList of Main path are combined.

FIG. 25 shows example displayed image showing how to mix displaying menubased on IG stream and video stream based on PlayList of Main path.

FIG. 26 shows example displayed menu based on IG stream and video streambased on PlayList of Main path.

FIG. 27 is a block diagram of a hardware, firmware, or software-basedcircuit structure use to control a first image and a second data image,such as a menu with selectable items.

FIG. 28 is a flow chart of how to control displaying.

FIG. 29 is another flow chart of how to control displaying.

FIG. 30 is a general block diagram of a reproducing apparatus.

BEST MODES FOR CARRYING OUT THE INVENTION

A brief description of the data structure of the BLU-RAY Read onlymemory disc recorded on ROM is defined in “BLU-RAY Disc Read-Only formatver1.0 part 3 audio visual specifications”. This data structure isdiscussed below as “BDMV”. Furthermore it is possible to record contenton a recordable disc, such as BD-R or BD-RE according to this BDMVformat.

For example, a bit stream encoded by the encoding method such as MPEG(Moving Pictures Experts Group) video or MPEG audio is called a Clip AVstream (or AV stream). The Clip AV stream is recorded on a discaccording to a file system called a “BLU-RAY Disc Read-Only Formatpart2”, or Clip AV stream file.

The Clip AV stream file is a management unit for a file system and isnot a management unit for a user to recognize easily. Considering auser's convenience, it is need to record database information on a discto achieve functions such as, to reproduce content, which are dividedinto a plurality of Clip AV stream files, to reproduce only a part of aClip AV stream file or to make a special playback operation in promptreply to a corresponding user operation. “The BLU-RAY Disc Read-Onlyformat ver1.0 part 3 audio visual specifications”, is one of the formatbooks of the BLU-RAY disc family, that provides such database.

Turning now to the figures, FIG. 1 indicates an abstract of a BLU-RAYROM data model. The BD-ROM data structure defines 4 layers as shown. Thelowest layer is a layer at which a Clip AV stream is disposed (from nowon referred to as Clip layer). The layer above the Clip layer is a layer(now referred to as PlayList layer) to dispose a Movie PlayList and aPlayItem to indicate the reproduction position of Clip AV stream. Thelayer above the PlayList layer is a layer (referred to as an objectlayer) to dispose items such as a Movie object including a command todesignate the reproducing order of a Movie PlayList. And the most upperlayer is a layer (referred to as an Index layer) to dispose an indextable to manage a title which is recorded on BLU-RAY disc.

The Clip AV stream is a bit stream that has video data and/or audio datamultiplexed in MPEG 2 TS (transport stream) form. The informationrelated to this Clip AV stream is recorded as Clip Information.

Furthermore, data streams for subtitles or menu displays that arepresented with video data or audio data, can be multiplexed in the ClipAV stream. A stream to present a subtitle is called as Presentationgraphic stream. A stream to present a menu is called as Interactivegraphic stream. However, they are equally well referred to as a stream.

A Clip AV stream file and clip information may be regarded as one objectand called a Clip. That is to say, Clip is an object that includes aClip AV stream file and a Clip information file.

Files are generally treated as a byte string. Contents in a Clip AVstream are spread over a time axis and an entry point in the clip isdesignated using a time-based point. When a user provides a time stampfor an access point in a clip, a reproducing apparatus can use a clipinformation file to know the address information at which the apparatusbegan to play corresponding to a provided time stamp.

The Movie PlayList has information designating Clip AV stream files toplay and has information designating a plurality of start positions (INtime) and end positions (OUT time) to indicate the reproductionpositions of the designated AV stream files. One set of these startpositions and end positions is called a PlayItem. A Movie PlayList hasone or more PlayItems. To play a PlayItem means to play at least part ofAV stream file referred by the PlayItem. That is to say, a section inthe Clip is played based on the IN time and OUT time information.

A Movie Object includes a HDMV Navigation command program (HDMV program)and terminal information. HDMV program is a command to controlreproduction of PlayList. Terminal Information includes permissioninformation to permit or restrict user interactive operation, such ascalling menu scene or title search that is restricted based on thisterminal information, to the BLU-RAY player.

A BD-J object includes a JAVA program Object.

The Index layer includes an index table. The Index table is a top layerand defines the BD-ROM title. The Module manager included in a BD-ROMstationed system software controls playing the BD-ROM disc based on thistitle information in the index table.

That is to say, as shown in FIG. 2 abstractly, an arbitrary entry in theindex table is called a title, First playback, Top menu, and title #1,#2 . . . . Registered in the Index table are all titles. Each title haslinks to Movieobject or BD-J object and indicates whether HDMV title orBD-J title.

For example, First Playback is a commercial image (trailer) playedbefore the substantial movie if the content in the BD-ROM is a movie.The top menu is a menu scene to select playback of a substantial movie,chapter search, subtitle, language or special contents. The Title can bea movie selected from the Top menu in this example and can be a Top menuor a sub menu.

FIG. 3 is a UML (Unified Modeling Language) figure showing therelationship between a Clip AV stream, Clip information (streamAttributes), Clip, PlayItem and PlayList. PlayList has a relationshipwith one or more PlayItems and a PlayItem has a relationship with aClip. One Clip can have a plurality of relationships with a plurality ofPlayItems, which have different IN time and/or OUT time from each other.Also, clip AV stream files and clip information files have one-to-onecorrespondence. By defining this data structure, the user can designatethe playback order of an arbitrary portion of the clip AV stream withoutdestroying and altering the Clip AV stream.

As shown in FIG. 4, a plurality of PlayLists can refer to a same clip.Furthermore one PlayList can refer a plurality of Clips. A Clip isreferred to by IN time and OUT time in PlayItem included in thePlayList. The example as shown in FIG. 4 indicates that Clip 300 isreferred by IN time and OUT time indicated by PlayItem 320 in PlayList310 and indicated by PlayItem 321 in PlayList 311 which has PlayItem 321and PlayItem 322. Further a section in Clip 301 is referred by IN timeand OUT time indicated by PlayItem 322 in PlayList 311, and anothersection in Clip 301 is referred by IN time and OUT time indicated byPlayItem 323 in PlayList 312 which has PlayItem 323 and PlayItem 324.

Furthermore, PlaytList can have a subpath corresponding to a SubPlayItemregarding to main path corresponding to PlayItem which played as maindata, as shown in FIG. 5. SubPlayItem can have a relationship withdifferent several clips, and SubPlayItems can choose selectively oneclip among a plurality of clips corresponding to SubPlayItem. A PlayListcan have a SubPlayItem under certain conditions.

Next, we explain file management information defined in “BLU-RAY DiscRead-Only format part 3” using FIG. 6. A file is managed using ahierarchical directory structure. At first one directory (root directoryin the example as shown in FIG. 6) is made on a recording medium. Arecording and/or reproduction system can managed under this directory.

Under root directory, “BDMV” directory and “CERTIFICATE DIRECTORY” aredisposed. “CERTIFICATE DIRECTORY” has information related to copyrightinformation. Under “BDMV” directory, data structure explained using FIG.1 is recorded.

Right under “BDMV” directory, “index.bdmv” file and “MovieObject.bdmv”file are disposed. Under “BDMV” directory, “PLAYLIST” directory,“CLIPINF” directory, “STREAM” directory, “AUXDATA” directory, “META”directory, “BDJO” directory, “JAR” directory, and “BACKUP” directory aredisposed.

“index.bdmv” file describes the contents of BDMV directory. That is tosay this “index.bdmv” file corresponds to an index table on an indexlayer which is a top layer.

“MovieObject.bdmv” file has one or more MovieObject information. That isto say, this “MovieObject.bdmv” file corresponds to the object layer.

“PLAYLIST” directory is a directory in which one or more PlayList filesare disposed. That is to say, the “PLAYLIST” directory includes a“xxxxx.mpls” file which is related to the Movie PlayList. “xxxxx.mpls”is a file corresponding to each PlayList.

The “xxxxx” before “.mpls” is a 5 digit number and the “mpls” portionafter the period is an extension fixed to this type of file.

“CLIPINF” directory is a directory in which one or more clip informationfiles are disposed. That is to say, a “CLIPINF” directory includes“zzzzz.clpi” files which are clip information files corresponding toeach Clip AV stream file. “zzzzz” before “.” is a 5 digit number and the“clpi” after the period is an extension fixed to this type of file.

“STREAM” directory is a directory in which substantial AV stream filesare disposed. That is to say, “STREAM” directory has Clip AV streamfiles each of which are corresponding to Clip information file. Clip AVstream file can have transport stream of MPEG 2 (Moving Pictures ExpertsGroup 2) (referred as MPEG2 TS), and the file name is “zzzzz.m2ts”. Asthe “zzzzz” portion before the period is same with the correspondingclip information file, a player can easily know the relationship betweenclip information file and this Clip AV stream file.

An “AUXDATA” directory is a directory in which a sound file for use inmenu displaying, font file, font index file and bit map file.“sound.bdmv” has sound data related to an interactive graphic streamapplication. The file name is fixed as “sound.bdmv”. “aaaaa.otf” filehas a font data used in a subtitle or BD-J application.

The “aaaaa” before the period is a 5 digit number and the “otf” portionafter the period is an extension fixed to this type of file.

The “META” directory has a Meta data file. The “BDJO” directory and the“JAR” directory contain the file related to a BD-J object. Further“BACKUP” directory has a back up file of one or more files describedabove.

FIG. 7 indicates the syntax of movie PlayList file “xxxxx.mpls” calledfrom the movie object. “type_indicator” field having 32 bits (4 bytes)data length indicates this file is a moviePlayList file.“version_number” field having 32 bits (4 bytes) data length indicatesthe version of this movie PlayList. “PlayList_start_address” fieldhaving 32 bits (4 bytes) data length indicates a starting address ofPlayList block. “PlayListMark_start_address” field having 32 bits (4bytes) data length indicates the starting address of PlayListMark.

The “ExtensionData_start_address” field having 32 bits (4 bytes) datalength indicates the starting address of ExtenstionData( ) block.“ExtensionData_start_address” field indicates the starting address ofExtenstionData( ) as relative bytes numbered from the first byte of“xxxxx.mpls” file. The relative bytes number starts from “0”. If this“ExtensionData_start_address” field value is “0”, then it indicates thatthere is no ExtensionData( ) block in this “xxxxx.mpls” file.

In the AppInfoPlayList( ) block, the type of PlayList reproductionrestriction is described. In PlayList( ) block, PlayList is described.In PlayListMark( ) block, jump points such as chapter jump points aredescribed. ExtensionData( ) block has a various data to play a contentbased on this PlayList.

FIG. 8 shows the syntax that indicates the example of PlayList( ) block.The “length” field indicates a data length from just after this “length”field to the end of PlayList( ) and has a 32 bit data length. A regionafter length field is reserved for future use. In “number_of_subPath”field indicates number of subpath included in this PlayList.

The number of PlayItem( ) blocks in which a PlayItem is included isdescribed based on the number indicated by number_of_PlayItems. SubpathOblocks are described based on SubPath( ) block. The count number basedon for loop is a identification of SubPath( ) block, SubPath_id.

Further, Subpath is used for not only audio data as after recording usedto record on a recordable disc, but for indicating a sub image which isplayed synchronously/asynchronously with a clip indicated by a PlayItemwhen 2 images are compound.

FIGS. 9A and 9B show a syntax to show an example of a structure ofPlayItem( ) block. The “length” field have a 16 bits data lengthindicates a data length from the right after this length filed to theend of this PlayItem( ) block.

The Clip_Information_file_name[0] field having 40 bits (5 bytes) datalength indicates the name of clip information file which is referred toby this PlayItem. This PlayItem read out clip information file indicatedthis Clip_information_file_name. Clip_codec_identifier[0] indicates thea formula of codec of Clip AV stream which is used by PlayItem in thisPlayItem( ) block.

The next 11 bit region is reserved for future use. 1 bit “is_multi_angleflag” indicates whether this PlayItem is for multi-angle contents ornot. Connection_condition having 4 bits data length indicates theconnection state between two PlayItem. “PlayItems. Ref_to_STC_id[0]”indicates discontinuity information of STC(System Time Clock) and have 8bits data length.

IN_time and OUT_time field indicates the playback range of Main clip AVstream and each have 32 bits data length. IN_time field indicates astarting point and OUT_time field indicates an end point.

UO_mask_table( ) is a table in which restriction of receiving user inputis set. PlayItem_random_access_flag indicates whether or not apermission to random access within this PlayItem is granted. Next 7 bitsdata is reserved for future use.

The “still_mode” field indicates whether an image displayed at end ofPlayItem in PlayItem( ) block is displayed as a still image or not. Ifthe value of the still mode field is “0x01”, then the still time fieldindicates the “still time”. If the value of Still_mode field is otherthan “0x01”, then this 16 bit region is reserved for future use.

If the “is_multi_angle” flag has a value to indicate that this PlayItemis corresponding to the multi-angle (“1”), then the structure to managethe multi-angle is described.

FIGS. 10A, 10B and 10C, and FIGS. 11A and 11B indicate the syntax of aexample of STN_table( ). This STN_table( ) defines such as streamsplayed based on this PlayItem and define combination of other streamsthat can be played back with video stream referred to by the PlayItemcomposed of Mainpath (Primary video stream).

The streams other than the primary video stream are for example audiostreams referred by PlayItem (Primary Audio stream), video stream andaudio stream included in a clip corresponding to sub path which iscorresponding to main path (secondary video stream and secondary audiostream) and subtitle stream.

More precisely, the STN_table( ) block defines the combination of asecondary audio stream and a primary audio stream, the combination ofthe secondary video stream and the secondary audio stream, andcombination of the secondary video stream and the presentation graphicsstream (PG) or text subtitle stream for Picture in Picture (PiP)application.

The “length” field has a 16 bit data length and indicates data lengthfrom right after this length field to the end of STN_table( ) block.Next to the length field, is a reserved area (reserved_for_future_use)having 16 bits data and number_of_primary_video_stream_entries having 8bits are disposed. The number_of_primary_video_stream_entries indicatesthe number of streams which have primary_video_stream_id and which isregistered in STN_table( ) block. The value of primary_video_stream_idis information to identify video streams.

The number_of primary_audio_stream_entries indicates the number ofstreams which have primary_audio_stream_ids and which are registered inSTN_table( ) block. The primary_audio_stream_id is information toidentify audio streams.

The number_of_PG_txtST_stream_entries indicates the number of streamswhich have PG_txtST_stream_id and which iare registered in theSTN_table( ) block. In this block, the presentation graphic (PG) stream,which is bit map subtitle encoded by run length code and text subtitlestream (txtST) are registered. The value PG_txtST_stream_id isinformation to identify these two subtitle stream.

The number_of_IG_stream_entries is registered in STN_table( ) block andindicates the number of IG_stream_entries. In this block Interactivegraphics streams are registered.

For example, an interactive graphics menu reproduced by interactivegraphics stream displays items such as a button, icon and thumbnail toreceive user operation. Interactive graphics menu means two type ofdisplaying method. One is a pop-up menu which turns on or off aninteractive graphic stream according to ON/OFF inputted operation byremote commander (or directly on the play-back unit). And the other isalways-on, which is displayed all the time.

The “number_of_secondary_audio_stream_entries” field indicates thenumber of stream which is given secondary_audio_stream_id and which isregistered in the STN_table( ) block.

The “number_of_secondary_video_stream_entries” field indicates thenumber of streams which are given “secondary_video_stream_id” and areregistered in STN_table( ) block.

The “number_of_PiP_PG_txtST_stream_entries” field indicates the numberof subtitle stream for PiP, which is given secondaryPip_PG_txtST_stream_id and is registered in the STN_table( ) block.

After the 40 bit reserved area, reserved_for_future_use, six “for loop”is disposed. In first for loop, stream_entries( ) block andstream_attribute( ) block are repeated number times indicated by thenumber_of primary_video_stream_entries using primary_video_stream_id asa variable. This first for loop gives the primary_video_stream_id whichincrease one by one from value “0” to each video stream specified bystream_entry( ) block. And by the stream_entry( ) block andstream_attirubute( ) block define each stream attribute, such as whichPID (Packet Identification) is allocated to the video stream or else.

In second for loop, stream_entries( ) block and stream_attribute( )block are repeated a number times indicated by the number_ofprimary_audio_stream using primary_audio_stream_id as a variable. Thissecond “for loop” gives the primary_audio_stream_id which increases oneby one from value “0” to each audio stream specified by stream_entry( )block.

In third for loop, the stream_entries( ) block and stream_attribute( )block are repeated a number times indicated by thenumber_of_PG_txtST_stream using PG_txtST_stream_id as a variable. Thisthird “for loop” gives the PG_txtST_stream_id which increase one by onefrom value “0” to each Presentation graphics stream or text subtitlestream specified by stream_entry( ) block.

In fourth for loop, the stream_entries( ) block and stream_attribute( )block are repeated a number times indicated by the number_of_IG_streamusing IG_stream_id as a variable. This fourth “for loop” give theIG_stream_id which increase one by one from value “0” to each IG streamspecified by stream_entry( ) block.

In fifth for loop, the stream_entries( ) block and stream_attribute( )block are repeated a number times indicated by thenumber_of_secondary_audio_stream_stream using secondary_audio_stream_idas a variable. This fifth “for loop” give the secondary_audio_stream_idwhich increase one by one from value “0” to each secondary audio streamspecified by stream_entry( ) block.

In fifth “for loop”, not only the stream_entries( ) andstream_attiribute( ) block, but alsocomb_info_Secondary_audio_Primary_audio( ) are repeated a number timeswhich is indicated by secondary_audio_stream_id.comb_info_Secondary_audio_Primary_audio( ) blocks indicate thecombination information of secondary audio and primary audio.

The comb_info_Secondary_audio_Primary_audio( ) block has anumber_of_primary_vaudio_ref_entries having 8 bits which indicate thenumber of primary audio referred by thiscomb_info_Secondary_vaudio_Primary_audio( ) block. Next to reserved area(reserved_for_word_ahgn) having an 8 bit data length, “for loop” isdisposed and “primary_audio_stream_id_ref” field is repeated a numbertimes which is indicated by “number_of_primary_vaudio_ref_entries”.Further if “number_of_primary_vaudio_ref_entries” is an odd number, thenanother reserved_for_word_align is disposed.

In six “for loop”, using value of secondary_video_stream_id as loopvariation, stream_entries( ) block and stream_attribute( ) block arerepeated a number of times as indicated bynumber_of_secondary_video_stream_entries, which is described above. Inthis for loop, secondary_video_stream_id is given to one video streamspecified by each stream_entry( ) in order.

In six “for loop”, the comb_info_secondary_video_primary_audio( ) blockand comb_info_Secondary_video_PiP_PG_textST( ) block are repeated anumber times indicated by the number_of_secondary_video_stream_entriesusing secondary_video_stream_id as a variable.

The Comb_info_Secondary_video_secondary_audio( ) indicates a combinationof Secondary video and secondary audio. Incomb_info_Secondary_video_Secondary_audio( ), thenumber_of_secondary_audio_ref_entries field have 8 bits indicatingsecondary audio stream number referred by thiscomb_info_secondary_video_secondary_audio( ) block. Further thenumber_of_primary_vaudio_ref_entries is an odd number, then anotherreserved_for_word_align is disposed as described in the next for loop.

The Comb_info_Secondary_video_PiP_PG_textST( ) block indicates thecombination of secondary video (ex. video for PiP) and PG and/or textST.In comb_info_Secondary_video_Pip_PG_textST( ), the“number_of_secondary_audio_ref_entries” field having 8 bits indicatesthe number of secondary audio stream number referred by thiscomb_info_Secondary_video_PiP_PG_textST( ) block. Further thenumber_of_PiP_PG_textST_ref_entries is an odd number, then anotherreserved_for_word_align is disposed as described in the next for loop.

By using this STN_table, one can not only define secondary video stream,primary audio stream, secondary audio stream and subtitle streamcombined with the primary video stream, but also prevent from thecontradictory combination of PG and text subtitle stream for Picture inPicture application.

Further the stream_attribute( ) block in from first “for loop” to six“four loop” indicates the stream attribution of one elementary streamspecified by each stream_entry( ) block.

FIG. 12 indicates an example of a syntax of stream_entry( ). The“length” field having 8 bits indicates data length from right after thislength field to end of this stream_entry. Next the “type” field has 8bits indicating stream type specified by this stream_entry( ) block.This example has three type values “1”, “2” and “3”. As described I thenext “if sentence”, the contents in this block can be changed accordingto the value of type field.

If the value of type field indicates 1, thenref_to_stream_PID_of_mainClip field having 16 bits is described. Theref_to_stream_PID_of_mainClip indicates the Packet ID (PID) to specifyone elementary stream from a plurality of elementary stream multiplexedin Main clip referred from PlayItem having stream_entry( ) block.

If the value of the type field indicates “2”, then ref_to_SubPath_idhaving 16 bits length and ref_to_subClip_entry_id having a 8 bit datalength and ref_to_stream_PID_of subClip having 16 bits data length aredisposed. “ref_to_SubPath_id” field indicates identification informationto identify subpath. ref_to_subClip_entry_id indicates referred clip.ref_to_stream_PID_of subClip field indicates PID to specify a elementarystream of clip referred by ref_to_subClip_entry_id used by Subpath. Thatis to say, if the value of type field indicates “2”, thenref_to_stream_PID_of subClip identifies one elementary stream from aplurality of clips which have a plurality of multiplexed elementarystream.

The stream_entry( ) defined in type field “2” is used when a pluralityof clips are referred in SubPlayItem( ) and these clips have a pluralityof multiplexed elementary streams.

If value of type is “3”, then ref_to_SubPath_id having a 8 bit datalength and ref_to_stream_PID_of_of_mainClip having a 16 bit data lengthare described. The ref_to_SubPat_id field indicates identificationinformation to identify subpath. ref_to_stream_PID_of_of_mainClipindicates PID of elementary stream included in main clip and referred bysubpath indicated by ref_to_SubPath_id. That is to say if type fieldindicates “3”, then this information specifies one elementary streamreferred by subpath from a plurality of elementary streams multiplexedin the main clip when a plurality of elementary streams are multiplexedin the main clip and the main path and subpath refer at least one ofelementary streams multiplexed in the main clip.

In stream_entry( ) block when the “type” field indicates “3”,ref_to_SubPath_id indicates SubPath_id and this SubPath_id identifySubpath (SubPath( ) block) defined in PlayList( ) as explained usingFIG. 8. Further by this Subpath calling SubPlayItem, we can manageIN_Time and Out_time of elementary stream.

Furthermore in the case that “type” field indicates “3”, a plurality ofelementary streams are multiplexed in a main clip and both main path andsubpath refer at least one elementary stream multiplexed in main clip.In the case SubPath_type field indicates “7” as described below.

As described above, by defining type value “1” and “2” in blockstream_entry( ) in a block STN_table( ), when a PlayItem and one or moreSubpath related to this PlayItem is prepared, we can identify oneelementary stream in a clip referred by PlayItem and clips referred byone or more Subpath. That is to say, when type field value indicates“1”, stream_entry( ) indicates Clip (main clip) referred by Main Pathand when type field value indicates “2”, stream_entry( ) indicates clip(Sub clip) referred by sub path.

As described above, by defining type value “1”, “2” and “3” in blockstream_entry( ) in a block STN_table( ), the player can identify oneelementary stream in a clip referred by not only PlayItem but alsoSubPlayItem, when a PlayItem and one or more Subpath related to thisPlayItem are described.

Next, block SubPath( ) in PlayList( ) is explained using FIG. 8 in moredetail. FIG. 13 shows syntax describing an example of block SubPath( ).The “length” field having a 32 bits data length indicates data lengthfrom right after this length field to end of this block SubPath( ).After reserved_for_future_use having a 8 bit data length, fieldSubPath_type is disposed. SubPatha_type indicates application type ofSubpath defined in block SubPath( ). For example according to value offield SubPath_type, Subpath defined in block Subpath( ) manages Audio,bitmap subtitle or text subtitle. This type of application is describedin more detail later.

After reserved_for_future_use, is_repeat_Subpath flag having 1 bit isdisposed. This is_repeat_Subpath flag indicates whether play Subpathdefined in SubPath( ) block repeatedly while main path is played or playSubPath only once. This is_repeat_SubPath indicates it is used forexample in case that playback timing of stream in the clip si referredby main path and stream in clip referred by subpath. One example is thatthe case main path indicates a slide show which the player plays as aplurality of still pictures while the player changes still picturesseamlessly and sub path indicates back ground music audio of the mainpath.

Next to “is_repeat flag” and reserved_for_future_use having a 8 bit datalength, number_of_SubPlayItems field is disposed.“number_of_SubPlayItems” field indicates number_of_SubPlayItem which asubpath defined in SubPath( ) block owns. SubPlayItem(i) blocks arerepeated as many times as indicated by number_of_SubPlayItems as shownin next “for loop”.

FIGS. 14A and 14B show an example of syntax of SubPlayItem(i) blockstructure. The “length” field has a 16 bit data length and right afterthis length field and SubPlayItem(i) block data length. Next fieldclip_information_file_name[0] has a 5 byte (40 bits) data length thatindicates file name of Clip information file. The value in the lastportion parenthesis ([ ]) indicates subclip_entry_id. This is commonamong similar description below. Clip_codec_identifier[0] having 4 bytes(32 bits) data length indicates system of CODEC of clip.

Next to “reserved_for_future_use” field, “is_multi_clip_entries” fieldhaving 1 bit is disposed. This “is_multi_clip_entries” field indicateswhether of not this SubPlayItem(i) refers to a plurality of clips. Ifthe value of is_multi_clip_entries is “1” (described as “1b”), then itindicates SubPlayItem refers to a plurality of clips.

The next field is ref_to_STC_id[0] having a 8 bit data length, andindicates information to relating to discontinuity relating system timeclock (STC). SubPlayItem_IN_time and SubPlayItem_OUT_time, each of whichhave 32 bit data lengths, and indicate playback start time and playbackend time. That is to say, these SubPlayItem_IN_time andSubPlayItem_OUT_time indicate playback section corresponding to Subpathin a clip.

Next the “sync_PlayItem_id” field having a 16 bit data length indicatesPlayItem played synchronously with Subpath defined by SubPlayItem(SubPlayItem(i)). sync_start_PTS_of_PlayItem has a 32 bit data lengthand indicates a playback start time of subpath on a time axis of mainpath. These sync_PlayItem_id and sync_start_PTS_of_PlayItem are not usedif a start time of a clip AV stream included in Main path and streamindicated by sub path are different.

If a value of is_multi_Clip_entries is “1”, then the description aboveis added further. Reserved area, reserved_for_future_use, having a 8 bitlength and num_of_Clip_enries are disposed. Num_of_Clip_entriesindicates clip number registered in subPlayItem(i).

As described in the next for loop, clip information is repeated a numbertimes as indicated in num_of_Clip_entries using SubClip_entry_id as avariable. Clip_information_file_name[subClip_entry_id] having a 5 byte(40 bits) data length indicates the file name of clip information fileto indicate clip. Clip_codec_identifier[subClip_entry_id] having 4 bytes(32 bits) indicates a system of CODEC of clip.ref_to_STC_id[subClip_entry_id] having a 8 bit data length indicatesinformation relating to discontinuity of system time clock (STC).

This “for loop” starts from the “1” of value of subclip_entry_id. On theother hand, descriptions above this if sentence (if(is_multi_clip_etries==1b)), then such as Clip_information_file_name[0],Clip_codec_identifier[0] and ref_to_STC_id[0] are described as value ofsubClip_entry_id is “0”. Further described above SubPlayItem_IN_time,SubPlayItem_OUT_time, Sync_PlayItem_id and sync_start_PTS_of_PlayItemare used for a plurality of clips referred by description in for loop.

FIGS. 15A and 15B show example of sub path types defined in SubPath( )block as SubPath_type. This example shown as FIGS. 15A and 15B definessub path type from value “2” to value “7”, and define value “0”, “1” andfrom “8” to “255” as reserved.

“Out-of-mux” as described in FIGS. 15A and 15B means that transportstream including elementary stream referred by sub path is differentfrom transport stream including elementary stream referred by main path.In another words, “Out_of_mux” means elementary stream referred by subpath is not multiplexed with elementary stream referred main path.Hereinafter, this “Out of mux” sub path type is referred to as a paththat is non multiplexed with Main path TS.

On the other hand, the “In-mux” as described in FIGS. 15A and 15B meansthat the transport stream including elementary stream referred by thesubpath is the same as the transport stream including elementary streamreferred by the main path. In another words, The “In_mux” means theelementary stream referred by the sub path is multiplexed with theelementary stream referred main path. Further, the “In-mux” means a clipof the AV stream of main and clip of the AV stream of sub are includedin the transport stream in the main clip. Hereinafter this “In-mux” subpath type is called a path that is multiplexed with Main path TS.

Further if the elementary stream referred by the main path is not playedwith the elementary stream referred by the sub path asynchronously, thenthis sub path type is called an asynchronous sub path. On the otherhand, if the elementary stream referred by the main path is played withthe elementary stream referred by the sub path asynchronously, then thissub path type is called a synchronous sub path.

A SubPath_type=“2” value indicates that the sub path is an audiopresentation path of a browsable slide show. A browsable slide showmeans that a still picture's slide show is played with musicasynchronously. For example, SubPath_type=“2” indicates path ofnon-multiplexed with Main path TS, and asynchronous path. That is tosay, audio presentation path referred by sub path and main path referredby PlayItem are asynchronous.

A SubPath_type=“3” value is particularly noteworthy because it indicatesthat the sub path is a path to display a presentation menu ofinteractive graphics. For example, SubPath_type=“3” indicates that subpath is a path of non-multiplexed with Main path TS and asynchronouspath. That is to say, a display of an interactive graphics streamreferred by sub path and main path referred by PlayItem areasynchronous.

A SubPath_type=“4” value indicates a presentation path of text subtitlestream. For example, SubPath_type=“4” indicates that sub path is a pathof non-multiplexed with Main path TS and synchronous path.

A SubPath_type=“5” value indicates that a sub path is a path ofnon-multiplexed with Main path TS and synchronous path. Further this Subpath refers at least one elementary stream (Primary Audio stream, PGstream IG stream and secondary audio stream) or Picture in Picturepresentation path.

The Picture in Picture presentation path means that a path of at leastone of primary audio stream, secondary video stream, secondary audiostream and sub title stream played corresponding to primary video streamwhich is a video stream referred by the main path.

As discussed in FIGS. 15A and 15B, a SubPath_type=“6” value indicatesthat sub path is a path of non-multiplexed with Main path TS, and is anasynchronous Picture in Picture presentation path which contains one ormore elementary stream paths. A SubPath_type=“7” value indicates thatsub path is a path of multiplexed with Main path TS, synchronous path,Picture in Picture presentation path which contains one or moreelementary stream paths.

An interactive graphics stream (IG stream) is now described. The IGstream is a data stream to display a menu of user-selectable items asdescribed above. For example, button images used for the menu displayare included in the IG stream. Further the IG stream defines a main pathTS non multiplexed type and main path TS multiplexed type. Main path TSnon multiplexed type can be a non synchronized path type, which meansthe sub path is played asynchronously with main path including primaryvideo.

The IG stream is multiplexed in a Clip AV stream, in either themultiplexed with primary video case or not. Interactive graphics stream(see FIG. 16A) has three segments, ICS (Interactive CompositionSegment), PDS (Palette Definition Segment) and ODS (Object DefinitionSegment).

ICS, one of three types of segments, has a basic structure informationof IG (which will be described in more detail later). PDS is a segmentwhich has color information of a button image. Further ODS is a segmentwhich has information of a Button shape. ODS has a button image itselfsuch as bitmap data to display button image compressed using run lengthcompression or other compression method.

ICS, PDS and ODS are divided into PES packets, and are held in a payloadof a PES packet distinguish by PID as shown example in FIG. 16C. As thePES packet is defined, the maximum size of the PES packet is 64 KB (kilobytes) and these ICS and ODS can be relatively big in size, ICS and ODSare divided and held in payload of PES packets. On the other hand,because a PDS's size is usually smaller than 64 KB, PDS for 1 IG streamcan be held in one PES packet. Identification information in PID of eachPES packet indicates which one is held (i.e. ICS, ODS or PDS).

Each PES packet is divided into Transport packets of MPEG TS (TransportStream) (FIG. 16D). A header of the TS packet have the PID whichcontains identify data in the TS packet.

FIG. 17 shows an example ICS structure. This ICS has asegment_descriptor( ) block, video_descriptor( ) block,composition_descriptor( ) block, sequence_descriptor( ) block,interactive_composition_data( ) block. The segment_descriptor( ) blockindicates this segment is ICS. The video_descriptor( ) block indicatesframe rate and image size of video displayed with this ICS. Thecomposition_descriptor( ) block indicates status information of ICS. Thesequence_descriptor( ) block indicates whether the ICS is spread among aplurality of PES packets.

In more detail, the sequence_descriptor( ) block indicates whether ICSincluded in this PES packet is lead ICS or rearmost ICS.

That is to say, when ICS data size is big, ICS is divided into aplurality of PES packets, whose maximum size is fixed at 64 KB. At thistime, and in this non-exclusive example, the preceding position of ICS(i.e. shown in FIG. 17 except interactive_composition_data( ) block) mayexist only in lead PES packets and rearmost PES packets. And this can beomitted in other PES packets. When the sequence_descriptor( ) blockindicates that this ICS is lead ICS and rearmost ICS, the player is ableto know ICS is carried in only one PES packet.

FIG. 18 shows an example of a syntax representing structure ofinteractive_composition_data_fragment( ). FIG. 18 showsinteractive_composition_data_fragment( ) as interacitive_composition( )block. The interacitive_composition_length has a 24 bit data length thatindicates the data length of interacitve_composition( ) block from thisfield. The “stream_model” field has a 1 bit data length that indicateswhether this stream is multiplexed with the other stream.

The “stream_model” value “0” indicates that the IG stream is multiplexedand other elementary streams may be multiplexed with interactivegraphics stream in the MPEG 2 transport stream. Stream model value “1”indicates that non-multiplexed and only interactive graphics streamexists in the MPEG2 transport stream. That is to say, interactivegraphics stream can be multiplexed with AV stream and only interactivegraphics stream can be composed of Clip AV stream.

The User_interface_model has a 1 bit data length that indicates whetherthe menu is displayed by this stream is a pop-up menu or an “always on”menu. A pop up menu is a menu that can be turn on/off by predeterminedinput means, such as a remote commander's button or a button on playeritself. On the other hand, the always on menu is a menu that cannot beturned on/off by a user operation. The user_interface_model value “0”indicates a pop-up menu, and “1” indicates an always on menu.Furthermore, the pop-up menu is permitted only when stream_model valueis “1” and interactive graphics stream is not multiplexed with otherelementary stream.

If the Stream_model value is “0”, then the sentence (afterstream_model==‘0b’, i.e. composition_time_out_pts, andselection_time_out_pts) is valid. The “composition_time_out_pts” fieldhas a 33 bit data length that indicates the time when menu display willbe turned off. The “selection_time_out_pts” field has a 33 bit datalength that indicates a time when a user operation to operate the menuwill not be possible. Time is described by PTS (Presentation Time Stamp)defined by MPEG 2.

A “user_time_out_duration” field has 24 bit data length that indicatesautomatic initialization time of menu display. A “number_oLpage” fieldhas a 8 bit data length that indicates page number of this menu, usinginitial number “0”. That is to say, when this menu has a hierarchystructure and a plurality of pages, then the number_of_page value isbigger than or equal to “1”. By repeating the next for loop a numbertimes indicated by number_of_page, each page for menu display isdescribed.

When a disc which has a data structure defined above is loaded, a playerneeds to translate a command described in such as Movie object read fromdisc to an intrinsic command for the player. The player has software tomake such translation in read only memory (ROM) installed in the player.This software is called as BD virtual player because it makes a playeract according to BD ROM standard by mediating between disc and player.

FIGS. 19A and 19B show a BD virtual player from a functional view. FIG.19A shows the player action when a disc is loaded. When disc is loadedin a player and the player make an initial access (Step S30), a registeris initialized to memorize one or more common parameters used for onedisc (Step S31). Next in Step S32, the program is loaded from disc andexecuted. “Initial access” means that the disc is played the first time,such as when the disc is inserted.

FIG. 19B shows an example of a player action when the playback useroperation is executed at the time the player does not play. On firstpause state (Step S40), playback user operation is made by remotecommander (UO: User Operation). When the playback user operation ismade, at first common parameters in registers are initialized (StepS41). Next step S42, playback phase of Movie Object is made.

FIG. 20 shows playback of the PlayList in a playback phase of a MovieObject. Now explained is the case when instruction to play content oftitle #1 is made by a User Operation. The player refers to an indextable as shown in FIG. 2, and gets an object number corresponding tocontents of title #1 in response to an instruction to play contents. Forexample, if object #1 is a number corresponding to title #1, then theplayer executes Movie Object #1.

An example as shown in FIG. 20, has a program written in movie object #1composed of 2 sentences. The player.plays PlayList#1 based on firstsentence, “Play PlayList(1)”. PlayList #1 has one or more PlayItems thatare played in order. When playback of all PlayItems in PlayList#1 isaccomplished, the player executes Movie object 1 again and secondsentence is executed. The second sentence is “jump TopMenu” in thenon-limiting example of FIG. 20, so this command is executed, then TopMenu is played as described in the second sentence of the Movie Object.

Now is explained the display image that is performed according to oneembodiment. This embodiment uses a plane structure as shown in FIG. 21.Moving picture (video) plane 10 is displayed at a rearmost plane andimage data (mainly video data) is displayed. Subtitle Plane 11 is on theMoving picture (video) plane 10 and Subtitle data is displayed.Interactive graphics plane 12 is a front plane and character data todisplay the menu and bitmap data of button data image is displayed. Onedisplay has these three planes.

Now the graphics plane 12 is described as an interactive graphics plane12.

Moving picture plane 10, subtitle plane 11, interactive graphics plane12 can be displayed independently. Moving picture plane 10 has 1920pixel×1080 lines, and a data length per one pixel is 16 bits and has asystem which is 4 (Luminance Y), 2 (color difference signal Cb), 2(color difference signal Cr) (from now referred to as YCbCr(4:2:2)).YCbCr(4:2:2) is a color system with luminance signal Y, Cb and Cr having8 bits per one pixel. Interactive graphics plane 12 and subtitle plane11 are planes with 1920 pixel×1080 lines and each pixel having 8 bits,and the color system is 8 bits color map address using 256 colors.

Interactive graphics plane 12 and subtitle plane 11 have 256 level alphablending. When mixing with another plane, non transparency can be set ata 256 level. It is possible to set non transparency to each pixel. Fromnow on, non transparency α is represented by the range (0≦α≦1). α=0means totally transparency and α=1 means totally non transparency.

On Subtitle plane 11 and interactive plane 12, PNG (portable networkgraphics) images or JPEG image, image encoded using run lengthcompression, or bitmap image data can be displayed. PNG format canhandle non transparency information (also called as alpha(α) data).Alpha blending is processed based on this non transparency information.Palette data up to 256 colors can be used and color factors (indexes) ofeach pixel are expressed by index table.

FIG. 22 shows a graphics process for how to mix three planes accordingto FIG. 21. The structure as shown in FIG. 22 can be achieved by anycombination of hardware and software. Video image data on Moving pictureplane 10 is supplied to 422/444 transform circuit 20. Color system ofvideo image data is transformed from YCbCr(4:2:2) to YCbCr(4:4:4) by422/444 circuit 20 and inputted to alpha blending circuit 21.

Image data of subtitle plane 11 is input into palette 22A and out putimage data as RGB(4:4:4). When non transparency is set to this imagedata, set non transparency α1 (0≦α1≦1) is outputted from palette 22A. InPalette 22A, palette information is memorized corresponding to the PNGformat and index number is referred using inputted 8 bit data as anaddress. Based on this index number, RGB(4:4:4) data having 8 bits datais outputted based on this index number.

FIG. 23 shows an example of palette table memorized in palette 22A.Three primary colors R, G and B and non transparency α, each having 8bits are allocated to each 256 color index number value [0x00]−[0xFF](0x means hexadecimal). Based on inputted PNG image data, player referpalette table in Palette 22A. Palette 22A output R, G, B and nontransparency α, corresponding to index value.

RGB data outputted from palette 22A is supplied to RGB/YCbCrtransformation circuit 22B, and is transformed into luminance signal Y,color difference signal Cb and Cr (from now we call YCbCr data.). Thisis for treating a common data structure in a mixing plane after thisprocess, therefore standardizing in a YCbCr data structure, which ismoving picture data structure.

YCbCr data outputted from RGB/YCbCr transform circuit 22B and nontransparency α is inputted into alpha blending circuit 23. In alphablending circuit 23, inputted YCbCr data is multiplied by nontransparency α1. The result of the multiplication is inputted into oneterminal of adder 24. Furthermore each luminance signal Y, colordifference signal Cb, Cr is multiplied by α1 in alpha blending circuit23. The logical complement of α1, (1−α1), is supplied into alphablending circuit 21.

In Alpha blending circuit 21, video data inputted from 422/444transformation circuit 20 is multiplied by the logical complement of α1(1−α1). The product is inputted into the other terminal of adder 24. Bydoing this, moving picture plane 10 and subtitle plane 11 are mixed. Thesum is inputted into the alpha blending circuit 25.

Image data of interactive graphics plane 12 is inputted into palette 26Aand outputted as RGB(4:4:4) data. When non transparency is set to thisimage data, set non transparency α2 (0≦α2≦1) is outputted from palette26A. RGB data outputted from Palette 26A is transformed into YCbCr databy RGB/YCbCr transform circuit 26B and standardized into YCbCr dataformat, which is the same with moving picture format. YcbCr dataoutputted from RGB/YCbCr transformation circuit is inputted into thealpha blending circuit 28.

When Image data used on interactive graphics plane 12 is PNG format,then non transparency data α2 can be set to each pixel in image data.Non transparency data α2 is inputted into the alpha blending circuit 28.In the alpha blending circuit 28, each luminance signal Y, colordifference signal Cb and Cr of YCbCr data inputted from RGB/YCbCrtransformation circuit are multiplied by α2. The product is inputtedinto one terminal of adder 29. Furthermore the logical complement of nontransparency data α2 (1−α2) is inputted into alpha blending circuit 25.

In alpha blending circuit 25, the sum from the adder 24 is multiplied bythe complement of non transparency data α2 (1−α2). The result of thealpha blending circuit 25 is inputted into another terminal of adder 29and is added to the result of alpha blending circuit 28.

For example, if non transparency α a of a region which does not have animage to be displayed is set to α=0 on subtitle plane or graphics plane,then the plane under the plane can be shown. By doing this, video imagedata on moving picture plane 10 can be shown as a background image.

Next we will explain one exemplary embodiment to accomplish the pausingof a video display, while maintaing the interactivity withuser-selectable items from the menu. This embodiment uses another clockto manage the IG stream to show the pop up menu which is not synchronouswith the main path for video. By doing this, even when video streambeing played based on the main path described in the PlayList is paused,a user operation relating to execution of operations associated withitems selected from a pop up menu can be accepted abnd performed.

What follows is an explanation of an example of mixing a menu display byIG stream with a video stream played based on main path described inPlayList using FIGS. 24, 25 and 26.

FIG. 24 shows a graphical user interface as an example of a menu displaycreated by the IG stream. In this example, the background image 200,button 201A, button 201B and button 201C are displayed by the IG stream.When right or left user operation by remote commander is made, thesebuttons 201A, 201B and 201C are to be selected state corresponding tothe operation. Furthermore when one button is in selected state so thata determination is made of a user-selected operation, the function oroperation associated with that button is executed by the player.

An explanation is now given of a mix of video data on moving pictureplane 10 with this menu display using FIG. 25. Non transparency a ofportion except menu display including pull down menu 202 in FIG. 24 isset to α=0, and interactive graphics plane 12 is mixed with movingpicture plane 10. By doing this, the player can display as shown in FIG.26 mixing video data as shown in FIG. 25 with menu display shown in FIG.24.

Video data displayed on moving picture plane 10 is based on the PlayItemof the main path being processed based on a STC (system time clock). Onthe other hand, the menu display on the graphics plane 12 is based on anon-synchronous sub path that is processed based on different clock froma clock of video data, a so called “free run clock”. By doing this, whenthe video image is based on the main path and the menu is displayed inone display, user operation determination, such as selection of abutton, turn on/off menu display can be achieved even when video data ispaused. In another words, it is easy and reasonable to use same STCclock to display the menu as STC is a clock to make standard clocksignal. But this embodiment uses two clocks, one is to manage video andanother is to manage the IG stream.

FIG. 27 shows an example of structure to control a display. The STCportion 400 creates the STC. The STC created by the STC portion 400 issupplied to one terminal 410B of a first switch portion 400 and issupplied to an IG decoder 402 based on selection of switch portion 400.Furthermore STC is supplied to second switch 411 and is supplied basedon selection of second switch 411. The free run clock portion 401creates clock CLK with an arbitrary frequency. Clock CLK is supplied toone terminal 410A of first switch 410 and is supplied to IG decoder 402based on the selection of switch 410.

IG decoder 402 decodes the IG stream based on the supplied clock CLK orSTC and makes the IG stream available to interactive graphics plane 12.In the same way, video decoder 403 decode the video stream of the mainpath based on the STC and makes the video stream in a state that is ableto be supplied to moving picture plane 10.

The controller 420 process corresponds to the operation based on anoperation portion such as remote control commander. For examplecontroller 420 controls switch portion 410, switch portion 411 andswitch portion 412 corresponding to operation. Furthermore controller420 supply a control signal to control IG decoder 402 to IG decoder 402via third switch 403. By changing switch portion 4120N/OFF, ON/OFF ofthe output from IG decoder 402 can be controlled. Furthermore the firstswitch is used to be supplied from the IG decoder. But needless to say,if the player uses only non synchronous type, then switch 410 is notnecessary as the non synchronous IG can be played only based on the freerun clock.

Now a method is explained for how to display both menu and video by wayof a flow chart shown in FIG. 28 and FIG. 29. In FIG. 28, when videostream based on main path of PlayList is begun, then at first subpathtype is checked referred from the PlayList in Step S10. That is to say,SubPath( ) block in PlayList( ) block in PlayList is checked in StepS10. It is checked whether SubPah_type in SubPath( ) block is equal to“3” and this sub path is a subpath to display presentation menu ofinteractive graphics.

In more detail, in Step S10, PlayItem( ) block in PlayList( ) block isreferred and STN table( ) block in this PlayItem( ) block is referred(FIGS. 9A and 9B). Furthermore stream_entry( ) block in STN_table( )block of IG stream is searched and type field value is obtained from thestream_entry( ) block. When type value is “1” or “3”, then it means thatclip of main path is referred, so the player can know that theSubPath_type field value is not “3”.

Furthermore the selection of the IG stream is made by the controller tocontrol inputted operation corresponding to user operation by the remotecommander. For example, the controller to control inputted operationgenerates IG_stream_id to identify IG stream corresponding to useroperation and sends IG_stream_id to controller which interprets thesyntax of PlayList. This controller refer STN_table( ) block is based onsent IG_stream_id and search corresponding stream_entry( ). Furthermore,IG_stream_id is set “0” as default value.

Type field value is “2”, then ref_to_SubPath_id in stream_entry( ) blockis obtained, and SubPath( ) block in PlayList( ) block corresponding toref_to_SubPath is referred and Subpath_type value in Subpath( ) block isobtained.

If SubPath_type value is determined that is equal to “3” in Step S10,then processing based on Step S11 is made. In Step S11, input terminal410A of first switch 410 is selected. Free run clock CLK is supplied toIG decoder 402 and third switch 412 is made ON state. Furthermore,Second switch 411 is made ON state and STC is supplied to video decoder403. By doing this, output from IG decoder 402 and video decoder 403 iscontrolled non-synchronously.

On the other hand, if SubPath_type value is determined not to be equalto “3” in Step S10, then processing based on S12 is made. In Step S12,input terminal 410B of first switch 410 is selected. Second switch 411is made ON state and STC is supplied to video decoder 403 and IG decoder402. By doing this, output from IG decoder 402 and video decoder 403 iscontrolled synchronously.

FIG. 29 shows an example of processing in the case that playing video ispaused by user operation, while playing video stream based on PlayListof main path. For example, a first instruction is made by user input topause (stop playing content temporarily) by user operation using anoperation unit. This instruction is passed to controller 420.Corresponding to this instruction, controller 420 checks SubPath_type( )referred by this PlayList and determines whether the value ofSubPath_type( ) is equal to “3” or not (Step S20). Because the processof Step S20 is the same with the process of Step S10 in FIG. 28, adetailed explanation is omitted.

When the player determines that SubPath_type value is equal to “3”, thenprocess of Step S11 is made. In Step S21, input terminal 410A in firstswitch 410 is selected. Free run clock CLK is supplied to the IG decoder402 and the third switch 412 is turned on. the Second switch 411 is alsoturned off.

By doing this, playing the video stream based on the PlayItem of mainpath is paused as second switch 411 turned off. And output and decode ofIG stream on IG decoder 402 as free run clock is supplied to IG decoder402.

Next step S22, the controller determines whether an instruction by useroperation to change the pause state, such as by selecting the playcommand, is made or not. If changing the pause state is made, thenprocess of Step S23 is made and goes back to S11 in FIG. 28. That is tosay, input terminal 410A is selected and free run clock is provided toIG decoder 402 and third switch 412 is turned on. Second switch 411 isturned on and the STC is supplied to video decoder 403.

On the other hand, in S22 if releasing the pause state is not made, thenprocess of Step S24 is made, and controller determines whetherinteractive user operation (UO) nor Navigation command is received ornot in Step S24. In the case that neither interactive UO and navigationcommands are not received, then the flow goes back to S22.

In Step S24, in the case that interactive UO or navigation command arereceived, then process goes to Step S25 and received interactive UO ornavigation command is executed.

Interactive UO is an instruction made by user operation using remotecontrol commander based on displayed menu by IG stream. For example,corresponding to an operation of an operation unit, an instruction toselect a button, instruction to determination or turning ON/OFF pop upmenu as instruction of interactive UO is sent from operation unit.

Furthermore, navigation command is a command to control playback basedon PlayList on object layer or PlayList layer. Navigation command is,for example, commands for selection of stream, change of spoken languageor subtitle, change of angle, change of Picture in picture. Navigationcommand is sent corresponding to interactive UO.

In FIG. 26, interactive UOs are shown for operation to display menu, toselect button 201A, 201B or 201C, to display pull down menu 202corresponding to selected button and to select and determine button203A, 203B or 203C. Furthermore navigation command allocated to button203B is sent when button 203B is determined by interactive UO. Suchdetermination whether or not interactive UO or navigation command arereceived, and process is made when received these interactive UO ornavigation command in Step S24 and Step S25.

Furthermore if the navigation command received in Step S24 is one of thecommands to change stream positions, such as chapter jump or changeplayback title, then release of pause (e.g., resume playing) is executedwithout instruction to release of pause. On the other hand, ifnavigation command is one of commands to change the spoken language orsubtitle format, change of angle and change of picture in picture, thenpause state is kept as these command do not change the stream position.

In Step S20, if SubPath_type is not equal to “3”, then process goes toS26. In S26, all switch portions (switch portion 410, switch portion 411and switch portion 412) are turned off. By doing this, playback of videostream based on PlayItem of main path is paused temporarily and stopdecoding IG stream and menu display is off state.

And when an instruction to release the pause state (Step S27) is sent,then, process flow goes to Step S28 and state of switch goes to a statein S12 as described. That is to say, input terminal 410B of first switch410 is selected. Second switch 411 is made to the ON state and STC issupplied to video decoder 403 and IG decoder 402. By doing this, outputfrom IG decoder 402 and video decoder 403 is controlled synchronously.

As described above, using independent free run clock to STC, and in thecase the sub path to play IG stream is non synchronous with the mainpath and is a subpath to refer stream which is not multiplexed intransport stream referred by PlayItem of main path, the IG decoder 402to decode the IG stream is operated based on the free run clock CLK. Bydoing this, even when user pause playback of video stream of main pathplayed based on STC, player can decode IG stream a user canexecuteoperations displayed by the menu by IG stream.

Furthermore, although it was explained that the controller determinesthat the IG stream is non synchronous type or not based on SubPath_typeby analyzing PlayList of main path, this is not to be construed as asole embodiment for carrying out this funtion. For example, it ispossible for a controller to determine this based on the stream itself.For example, if only one subpath is defined corresponding to the mainpath and the stream corresponding to this Sub path is an IG stream, thenthe controller can determine that the player can play streamcorresponding to the sub path as a non synchronous type.

Next, we explain the reproduction apparatus which can play aboveexample. FIG. 30 shows an example of reproducing apparatus 1. Apparatus1 has storage drive 50, switch circuit 51, AV decoder portion 52 andcontroller 53. The storage drive is an optical drive for BD-ROM disc.Furthermore storage drive may optional include a hard disc or nonvolatile memory storing content data also.

The controller 53 includes a CPU (central Processing Unit), ROM (readonly memory) which has pre stored program so as to comprise a computerprogram product, and RAM (Random Access Memory) to be used as a workingmemory when executing the program and controlling the overall operationof this reproduction apparatus 1. Controller 420 as explained using FIG.27 is corresponding to this controller 53.

Reproduction apparatus 1 also has a user interface to send/receive useroperation and output control signal corresponding to receive useroperation. For example a remote commander to communicate withreproducing apparatus via infrared light or other communicating means isused as a user interface. Several input means are established on remotecontrol commander, such as cross button to indicates up, down, right andleft, number key, function key to execute predetermined function, suchas pause, stop, fast forward and back forward. As an alternative, theuser interface includes a set of buttons, and an optional display on theplayer itself.

Remote control commander generate control signals corresponding to theseoperations using such input mechanisms, and sends these control signalswirelessly such as by modulating an infra red signal. Reproductionapparatus 1 receives this infra red signal via a infra red signalreceiver (not shown, or optionally included in the controller) andtransforms this infra red signal into an electrical signal anddemodulates it into the original control signal. This control signal issupplied to controller 53. The controller 53 then controls thereproduction apparatus 1 corresponding to control signal based on theexecution of the program stored on ROM.

User interface can be not only remote control commander but also someswitches on operation panel on reproduction apparatus 1. Furthermore,reproduction apparatus 1 can have a communicating mechanism using a LAN(Local Area Network) and supply signal from an external computer viathis communicating mechanism to controller 53.

Furthermore initial information of language setting is memorized in aregister such as non-volatile memory. This initial information oflanguage setting is supplied to controller 53 from memory read when userturns on reproduction apparatus 1.

When a disc is inserted in storage drive 50, controller 53 reads“index.bdmv” and “MovieObject.bdmv” from the disc via storage drive 50.Controller 53 reads the PlayList file on “PlayList” directory based ondescription of these read file, and reads information relating to mainpath, sub path, and STN_table( ) block. Controller 53 reads the clip AVstream referred by PlayItem included in PlayList file from the disc viathe storage drive 50. Furthermore controller 53 read clip AV streamreferred by SubPlayItem included in PlayList file from disc via storagedrive 50 in the case the PlayList file includes SubPlayItem.

Controller 53 analyzes information in the PlayList file and judges thesubpath type based on SubPath_type corresponding to main path, and basedon judgment, selects whether to operate the interactive graphics decoder77 based on STC or free run clock CLK.

Hereinafter, the clip AV stream corresponding to the SubPlayItem isdescribed as a sub clip AV stream and the call clip AV streamcorresponding to PlayItem as main clip AV stream.

Data from storage drive 50 is processed through demodulation and errorcorrection processing by modulation portion and error correcting portion(not shown in figure) and restored in a multiplexed stream. Thismultiplexed stream is a time divided and the multiplexed transportstream which includes a plurality of transport packets with fixed datalength and data type identified by PID. This multiplexed stream issupplied into switch 51. Controller 53 classifies transport packetsbased on PID and control switch 51. Packets of the main clip AV streamare supplied into buffer 60 and Packets of sub clip AV stream aresupplied into buffer 61. Packets of sound data are supplied into buffer62. Packets of test data are supplied into buffer 63.

Packets stored in buffer 60 are read from buffer 60 based on control ofcontroller 53 and supplied to PID filter 64. PID filter 64 classifiesvideo packets, packets of PG stream, packets of IG stream and packets ofaudio stream.

In similar way, packets stored in buffer 61 are read from buffer 61based on control of the controller 53 and supplied to PID filter 90. PIDfilter 90 classifies video packets, packets of PG stream, packets of IGstream and packets of audio stream.

Video packet separated by PID filter 64 and Video packet separated byPID filter 90 are supplied into PID filter 65 and classified based onPID. That is to say, PID filter 65 supply packets of main clip AV streaminto 1^(st) video decoder 69 and supply packets of sub clip AV streaminto 2^(nd) video decoder 72.

1^(st) video decoder 69 extracts video data from a payload of a packet,and decodes it according to a predetermined decompression method, suchas MPEG 2 and H264/AVC. Output from the 1^(st) video decoder 69 issupplied to a 1^(st) video plane generator 70 and generates a videoplane. The video plane is generated by recording one frame of digitalvideo data into frame memory. The video plane generated by the 1^(st)video plane generator 70 is supplied to a video data processor 71.

2^(nd) video decoder 72 and 2^(nd) video plane generator 73, whichfollow similar processes to those described above with respect to the1^(st) video decoder 69 and 1^(st) video plane generator 70, generatevideo plane. The video plane generated in 2^(nd) video plane is suppliedinto video data processor 71.

Video decoder 403 explained using FIG. 27 corresponds to 1^(st) videodecoder 69. Video decoder 403 can correspond to 1^(st) and 2^(nd) videodecoder 69, 72. Furthermore in this figure STC 400 and switch 411 inFIG. 27 is omitted.

Video data processing module 71 generates a composite video plane thatcombines the video plane generated by the 1^(st) video plane generator70 and the video plane generated by 2^(nd) video plane generator 73 intoone frame. It is possible to make a video plane using video planesgenerated by 1^(st) video plane generator 70 or video plane generated by2^(nd) video plane generator 73 selectively. This video plane iscorresponds to video plane 10 shown in FIGS. 9A and 9B.

Packets of the PG stream are classified by PID filter 64 and PID filter90 and are supplied to switch circuit 66, where packets are selected andsupplied to presentation graphics decoder 74. Presentation graphicsdecoder 74 extracts PG data from the payload of packets and decodes itto generate graphics data to represent subtitles to be supplied toswitch circuit 75.

Switch circuit 75 selects one of graphics data and text subtitle dataand supplies the same to presentation graphics plane generator 76.Presentation graphics plane generator 76 generates the presentationgraphics plane and supplied it to video data processing 71. Thispresentation graphics plane corresponds to the subtitle plane describedin FIGS. 9A and 9B as an example.

Packets of the IG stream are classified by PID filter 64 and PID filter90 are supplied to switch circuit 67 and packets from PID filter 64 orfilter 90 selected and supplied to interactive graphics decoder 74.Interactive graphics decoder 77 reconstructs PES packets from extracteddata from payload of supplied packets. And based on header information,ICS, PDS and ODS of IG stream are extracted. ICS and PDS is stored inbuffer called as CB (composition buffer). ODS is memorized in buffercalled as DB (Decoded Buffer). For example preload buffer 78 correspondsto CB and DB.

Furthermore, PES packets have PTS which is time manage informationrelating to playback output and DTS, which is time managementinformation relating to decoding. Menu display by IG stream is displayedbased on PTS included in PES packet. For example, each data comprised ofIG stream and memorized in preload buffer is controlled to be read ontime based on PTS. One of STC and free run clock CLK is used as timeinformation which PTS is based on. When the free run clock CLK is usedas the basis for PTS, then the menu display is asynchronous with outputof 1^(st) and 2^(nd) video decoder 69 and 72.

Data from IG stream read from pre-load buffer 78 is supplied tointeractive graphics plane generator 79 and interactive graphics planeis generated. Interactive graphics plane corresponds to the graphicsplane 12.

Video data processor 71 has a graphics processor as explained in FIG.22, and mixes supplied video plane (moving picture plane 10 in FIG. 22),presentation graphics plane (subtitle plane 11 in FIG. 22) andinteractive graphics plane (graphics plane 12 in FIG. 22) to create onecomposite image data and output video signal for display on a display,or Television.

Audio streams classified by PID filter 64 and 90 are supplied to switchcircuit 68. Switch circuit 68 selects one of these supplied streams andsupplies one to 1^(st) audio decoder 80 and the other to 2^(nd) audiodecoder 81. Audio streams decoded by 1^(st) audio decoder 80 and 2^(nd)audio decoder 81 are mixed by adder 81 and further are mixed by adder 82with sound data read from buffer 62.

Text data read from buffer 63 is processed by text-ST composition and issupplied to switch 75.

Although the present description used many hardware features to describefunctionality, it should be understood this reproduction apparatus canbe implemented more or all software implemented on a general purpose, orspecial purpose hardware platform. In the case of the reproductionapparatus being implemented on a computer, the apparatus includes both ahardware portion and a software portion. For example some decoders onreproduction apparatus 1, especially 1^(st) video decoder 69 and 2^(nd)video decoder 72 are processor intensive and so it is possible to makesome processes which require high processing ability as hardware, andmake other parts as software.

The programs that run on a computer to make a reproduction apparatus 1comprising only software or a hybrid mix of hardware and software may beprovided by, such as distributing CD-ROM (Compact Disc Read OnlyMemory), DVD-ROM (Digital Versatile Disc Read Only Memory), BD-ROM(BLU-RAY Disc Read Only Memory) and by using the internet. Onceinstalled on the computer, the computer can peform predeterminedprocessing as described above. Any one of the SONY VAIO (Trademark)personal computers could serve as the host for such software operations.

DESCRIPTION OF REFERENCE NUMERALS

-   -   1 REPRODUCING APPARATUS    -   10 MOVING PICTURE PLANE.    -   11 SUBTITLE PLANE    -   12 INTERACTIVE GRAPHICS PLANE    -   201A, 201B AND 203C BUTTON    -   202 PULL DOWN MENU    -   203A, 203B AND 203C BUTTON    -   400 STC PORTION    -   401 FREE RUN CLOCK    -   402 IG DECODER    -   403 VIDEO DECODER    -   410 FIRST SWITCH    -   411 SECOND SWITCH    -   420 CONTROLLER

The invention claimed is:
 1. A reproducing apparatus, comprising: areference signal unit configured to output a reference signal; anacquiring unit configured to acquire a first stream of data thatrepresents a main image to be reproduced on a display under control ofsaid reference signal, a second stream of data that represents a menu tobe displayed with said main image, said second stream being asynchronouswith said first stream, and said menu including an item that isuser-selectable; and reproducing control information; an operationreceiving unit to receive an operation request when the item in saidmenu is selected; and a reproducing control unit configured to controlreproduction of said main image from said first stream, and the menufrom said second stream based on said reproducing control information,wherein the reproducing control unit is configured to display the mainimage as a video and to simultaneously display the menu; and saidreproducing control unit is configured to pause reproduction of thevideo in response to acquiring a first portion of the reproducingcontrol information, and to control the operation receiving unit toexecute, while the video is paused, a computer-implemented operation inresponse to the item in said menu being selected.
 2. The reproducingapparatus according to claim 1 wherein: said reproducing controlinformation includes information indicating a characteristic descriptionof another stream corresponding to said first stream; and saidreproducing control unit is configured to determine whether said anotherstream is said second stream.
 3. The reproducing apparatus according toclaim 1, further comprising: a clock signal outputting unit configuredto output a clock signal that is independent of said reference signal,wherein said reproducing control unit is configured to control saidsecond stream using said clock signal at least when said operatingreceiving unit receives as an operation request a request to pausereproduction of said main image.
 4. The reproducing apparatus accordingto claim 3, wherein: said first stream of data is stored on an opticaldisc and represents a high definition video image; said second stream ofdata is stored on the optical disc and represents the menu; and saiditem is at least one of a subtitle and a language setting.
 5. Acomputer-implemented method of reproducing an image and a menu on adisplay comprising: outputting a reference signal, acquiring a firststream of data that represents a main image to be reproduced on thedisplay under control of said reference signal, a second stream of datathat represents a menu to be displayed with said main image, said secondstream being asynchronous with said first stream, and said menuincluding an item that is user-selectable; and reproducing controlinformation; reproducing said main image from said first stream, andreproducing said menu from said second stream based on said reproducingcontrol information; displaying said main image as a video, andsimultaneously displaying said menu; pausing said video in response toacquiring a first portion of said reproducing control information; andselecting the item in said menu in response to acquiring a secondportion of said reproducing control information, and executing, whilethe video is paused, a computer-implemented operation in response to theitem being selected in said selecting step.
 6. The method according toclaim 5, wherein said reproducing control information includesinformation indicating a characteristic description of another streamcorresponding to said first stream, and said method further comprising:determining whether said another stream is said second stream.
 7. Themethod according to claim 5, further comprising: outputting a clocksignal that is independent of said reference signal; and controllingsaid second stream using said clock signal at least when said pausingstep pauses the video.
 8. The method according to claim 7, wherein: saidacquiring step includes acquiring said first stream of data from anoptical disc; said displaying step includes displaying said main imageas a high definition video image; said acquiring step further includesacquiring said second stream of data from said optical disc; and saiditem being at least one of a subtitle and a language setting.
 9. Anon-transitory recording medium on which computer readable instructionsare encoded such that when executed by a processor implement a methodcomprising: outputting a reference signal, acquiring a first stream ofdata that represents a main image to be reproduced on the display undercontrol of said reference signal, a second stream of data thatrepresents a menu to be displayed with said main image, said secondstream being asynchronous with said first stream, and said menuincluding an item that is user-selectable; and reproducing controlinformation; reproducing said main image from said first stream, andreproducing said menu from said second stream based on said reproducingcontrol information; outputting said main image as a video, andsimultaneously outputting said menu for display on a display; pausingsaid video in response to acquiring a first portion of said reproducingcontrol information; and selecting the item in said menu in response toacquiring a second portion of said reproducing control information, andexecuting, while the video is paused, a computer-implemented operationin response to the item being selected in said selecting step.
 10. Therecording medium according to claim 9, wherein said reproducing controlinformation includes information indicating a characteristic descriptionof another stream corresponding to said first stream, and said methodfurther comprising: determining whether said another stream is saidsecond stream.
 11. The recording medium according to claim 9, furthercomprising: outputting a clock signal that is independent of saidreference signal; and controlling said second stream using said clocksignal at least when said pausing step pauses the video.
 12. Therecording medium according to claim 11, wherein: said acquiring stepincludes acquiring said first stream of data from an optical disc; saidoutputting step includes outputting said main image as a high definitionvideo image; said acquiring step further includes acquiring said secondstream of data from said optical disc; and said item being at least oneof a subtitle and a language setting.
 13. A reproducing apparatus,comprising: a reference signal unit configured to output a referencesignal; an acquiring unit configured to acquire a first stream of datathat represents a main image to be reproduced on a display under controlof said reference signal, a second stream of data that represents a menuto be displayed with said main image, said second stream beingasynchronous with said first stream, and said menu including an itemthat is user-selectable, and reproducing control information; means forreproducing said main image from said first stream, and for reproducingsaid menu from said second stream based on said reproducing controlinformation; a video data processor configured to output said main imageas a video and to output simultaneously said menu for display on adisplay, wherein the means for reproducing is configured to pause saidvideo in response to acquiring a first portion of said reproducingcontrol information, to select the item in said menu in response toacquiring a second portion of said reproducing control information, andto execute, while the video is paused, an operation in response to theitem being selected.
 14. The reproducing apparatus according to claim 13wherein: said reproducing control information includes informationindicating a characteristic description of another stream correspondingto said first stream; and said means for reproducing determines whethersaid another stream is said second stream.
 15. The reproducing apparatusaccording to claim 13 further comprising: a clock signal outputting unitthat is independent of said reference signal, wherein said means forreproducing reproduces said second stream using said clock signal atleast when a request to pause reproduction of said main image isreceived.
 16. The reproducing apparatus according to claim 15, wherein:said first stream of data is stored on an optical disc and represents ahigh definition video image; said second stream of data is stored on theoptical disc and represents the menu; and said item is at least one of asubtitle and a language setting.